[IMP] common: halve the time odev takes to start and exit - #176
Merged
Conversation
Every invocation paid ~1.6s of framework overhead before the requested command even began, and kept the user waiting ~0.35s after it had already printed its result. For a tool run dozens of times a day that overhead is the dominant part of how slow odev feels, and none of it was the command itself: `odev version` measured 1.59s median, of which 0.02s was the command. Five causes, all of them paid unconditionally: - Telemetry blocked the main thread on an HTTPS round-trip in `run_command`'s `finally`. Records are now spooled locally when a command ends and submitted in the background by a later run, so no invocation ever waits on the endpoint. This also fixes `Telemetry.update()`, which never ran: its `len(_command_stack) != 1` guard was evaluated after the stack had been popped, so exit codes and execution times were silently dropped. The employee check, which ran a vault lookup, an SSH-agent decryption and a call to git on every command, is now cached in the configuration. - Scanning the sources for interactive debuggers spawned two `grep` processes at import time. It now runs on demand and its result is cached on disk, keyed on a walk of the command trees that costs a fraction of the scan itself. - The GitHub API client, paramiko, InquirerPy and black were imported by the framework although only a handful of commands need them; the connector and mixin packages re-exported everything they contained. Those imports moved to their point of use and the packages resolve their names lazily. Networkx was only used to sort the plugin dependency graph and is replaced by a topological sort, reporting the same cycles. - Discovering commands executed all 46 command modules only to read their names, so every run paid for every command, plugins included. Names, aliases and help texts are now cached in an index and a command module is only imported once that command runs. Plugin patching is replayed from the recorded registration order, leaving the output of `odev help` unchanged. - `CaptureOutput` attached its handlers to the loggers that existed when it was entered, which no longer covers command modules imported while a command runs. It now captures on the root logger, like odev does in production. `odev version` goes from 1.59s to 0.73s median, measured by interleaving runs against a clean checkout, and imports 1022 modules instead of 1503. `tools/benchmark_startup.py` reports the breakdown, and the new tests guard the structural causes rather than timings, which are too noisy to assert on. Claude-Session: https://claude.ai/code/session_012tWyDuUYenE92nsBtG19Du
The first run of a new version has no index yet and legitimately imports every command to build one, so the startup tests only passed on a machine where a previous run had already warmed it up. On a clean checkout they measured the cold path and failed, and test_03 only passed because test_02 happened to run first and leave an index behind. Claude-Session: https://claude.ai/code/session_012tWyDuUYenE92nsBtG19Du
The script carries a shebang and is meant to be run directly, which ruff enforces through EXE001. The mode bit went unnoticed locally because the working tree lives on a filesystem that does not report it faithfully. Claude-Session: https://claude.ai/code/session_012tWyDuUYenE92nsBtG19Du
Contributor
Author
|
The follow-up mentioned at the end of the description is now open: odoo-odev/odev-plugin-ai-upgrade#9. It removes the With both changes applied,
This PR stands on its own — #9 is not required for it to merge, and the reverse is also true. |
sea-odoo
previously approved these changes
Jul 30, 2026
brinkflew
added a commit
that referenced
this pull request
Aug 6, 2026
Merging beta brings in the deferred imports of #176, which moved the GitHub client out of the module scope. The GithubConnector extracted here keeps them inside the methods that need them, and the connectors package registers it with the lazy exports instead of importing it eagerly. Also bumps the version to 4.31.0, one increment above beta.
brinkflew
added a commit
that referenced
this pull request
Aug 6, 2026
odev.common.logging configures logging on import, but logging.basicConfig is a no-op once the root logger has handlers: whether odev's handler gets installed depends on whether that import happens before or after pytest sets its own up, and importing the sandbox from conftest tips it. Combined with the capture added by #176 every record then reached the output twice, and a plain logger.info became a console.print that #177's run_hook test asserts on. The handler is dropped in pytest_configure so the suite no longer depends on import order. Also resolves the version command test, which this branch made namespace-agnostic while #175 rewrote it, and bumps the version to 4.31.2, one increment above the base branch.
This was referenced Aug 6, 2026
brinkflew
added a commit
to odoo-odev/odev-plugin-ai-upgrade
that referenced
this pull request
Aug 27, 2026
Odev imports every plugin package on startup so that plugins can patch the framework before commands are registered. This one additionally pulled in its own `commands` subpackage, which imports `upgrade`, which imports networkx: every odev invocation paid for it, including the ones that have nothing to do with this plugin. Nothing needed those imports. Odev discovers commands itself, loading `commands/*.py` directly rather than through the package, and this plugin patches nothing on load: its `common/__init__.py` is empty. An empty `__init__.py` matches the four sibling plugins that do not patch either. Together with odoo-odev/odev#176 this removes the last expensive import from odev's startup path: 1022 modules down to 714, and `odev version` from 0.688s to 0.503s. Claude-Session: https://claude.ai/code/session_012tWyDuUYenE92nsBtG19Du
brinkflew
added a commit
to odoo-odev/odev-plugin-ai-upgrade
that referenced
this pull request
Aug 27, 2026
`commands/upgrade.py` imports networkx to sort modules by dependency, but only copier and jinja2 were declared: the package happened to be installed because odev core depended on it. odoo-odev/odev#176 removed core's last use of networkx, so this plugin is now its only consumer and cannot keep relying on someone else pulling it in. Claude-Session: https://claude.ai/code/session_012tWyDuUYenE92nsBtG19Du
brinkflew
added a commit
that referenced
this pull request
Aug 27, 2026
…st suite (#178) ## Why This started as filling coverage gaps, and each step surfaced the next: 1. `.coveragerc` gates at 60% and the suite sat at **64%**, with the gap widest on pure logic the rest of the framework leans on. Writing those tests surfaced **six defects in the helpers** — without fixing them the tests would have pinned broken behaviour. 2. Verifying the fixes meant running the suite repeatedly, which is when it became clear that **two suites cannot run at once**: identical invocations produced anywhere from 0 to 68 failures, and the machine had accumulated 16 orphaned `/tmp/odev-test-*` directories. 3. Isolating the runs revealed that three concurrent suites exhaust PostgreSQL's connection slots, which turned out to be a **connection leak in odev itself**, not in the suite. The four sections below are independent and the commits are ordered to be reviewed in sequence. ## 1. Helper defects — `42569c4` | Location | Defect | |---|---| | `connectors/postgres.py` `columns_exist` | Returned `[]` when **none** of the requested columns existed — indistinguishable from all being present. `CREATE TABLE IF NOT EXISTS` leaves an existing table alone, so the missing-columns pass is the only thing that can migrate a table created from an older definition; it silently added nothing. | | `postgres.py` `PostgresDatabase.tables` | A class attribute, so every instance shared one registry and tables from different databases collided on their name alone. | | `string.py` `quote` | Chose its delimiter with `max()` over both quote offsets, picking the **last** rather than the first, mis-quoting any string mixing them. | | `version.py` `OdooVersion.__bool__` | Always `True` — `module` is padded to `MIN_VERSION_LENGTH` and is never an empty tuple. | | `string.py` `min_indent` | Raised `ValueError` on a text without any non-blank line, reachable from `odev help` through `dedent`. | | `float_to_hours`, `strip_styles` | Broken, but called nowhere in odev nor in the plugins. **Left alone**, documented in the tests with the correction spelled out. | `columns_exist` has exactly one caller, and it runs after `CREATE TABLE IF NOT EXISTS`, so the fix cannot make it issue `ALTER TABLE` against a missing table. ## 2. Coverage for the untested helpers — `be95197` - **`test_string.py`** (new) — `string.py` had no test module at all, despite backing `odev help`, `odev history` and the local database listing query. Sizes and their round-trip, indentation, joining, the `dirty_only` × `force_single` quoting matrix, Rich markup helpers, and the `help` column alignment contract. - **`test_git_worktree.py`** (new) — `connectors/git.py` was the least-covered large module (34%), and its `GitWorktree` parser turns `git worktree list --porcelain` into the objects the whole `fetch` / `pull` / `worktree` family works with. Porcelain parsing (branch, detached, bare, locked, prunable with reasons), the `-odev-` local-branch split that `create_worktree` writes and `fetch` / `pull` read back, identity by path, and `pending_changes` including the two swallowed `GitCommandError` messages. No network, no real repository. - **`test_postgres_table.py`** (new) — `PostgresTable.__add_missing_column`, the datastore's migration path, was entirely unreached; this covers it including the `InvalidTableDefinition` primary-key branch. - **`test_version.py`** — ordering (`15.0 < 16.0 < saas-16.4 < 17.0 < master`) is what actually picks a revision at runtime and nothing compared two versions. Corrections to existing tests, in the same commit: - **`test_bash.py` shelled out to a real `sudo cat >> /etc/shadow`.** The premise that the command fails only holds for an unprivileged user whose shell cannot open the redirection — a machine granting passwordless sudo runs it for real, and as root it appends to the file or hangs on stdin. The subprocess and the effective user are now simulated, which also lets the elevation path be asserted rather than inferred. - `test_odev.py` left a command line behind in `sys.argv` for whichever test ran next. - `tests/fixtures/case.py` — `_patches` was a list defined on `OdevTestCase` and mutated through `cls._patches.append`, so every subclass shared it and each class tore down the patches of all the classes before it. ## 3. An isolated, self-cleaning test suite — `87dc0e4`, `86630d3`, `716c8e1` `Odev.name` was the constant `"odev-test"` and **every** shared resource derived from it, so two suites shared one namespace and actively destroyed each other: - `test_99_delete_expression` ran `odev delete --expression "^odev-test-[a-z0-9]{8}" --include-whitelisted` against the real PostgreSQL, deleting a concurrent run's databases. - `PostgresDatabase.drop()` terminates every backend on `datname`, so each class teardown killed a concurrent run's cursors. - `CREATE TABLE IF NOT EXISTS` is not atomic, and `Config.save()` truncate-writes a fixed path — hence `UniqueViolation` on `pg_type_typname_nsp_index` and `DuplicateOptionError` from a torn config. A run now claims a sandbox named after itself and holds an exclusive `flock` on it for its whole life. Everything — datastore, test databases, config, temp directories — is named after it or nested under it. Cleanup runs at `pytest_sessionfinish`, which pytest calls from a `finally`, so `Ctrl+C` is covered; `SIGTERM` becomes the same orderly exit; and the next run's sweep collects whatever a `SIGKILL` left, because the kernel releases the lock when the owner dies whatever the cause. **The suite was also writing outside its sandbox**, which is worth a look on its own: - `TestSetup` ran the install scripts against their real destinations, so running the suite **repointed the developer's `~/.local/bin/odev` and bash-completion symlinks at whichever checkout it ran from**. `symlink.py` computed the destination halfway through creating it, leaving no way to redirect it; that decision moves to `link_path`. - Tests cloned into the real `~/odoo/repositories`. The repositories, dumps and upgrade paths now point inside the sandbox, as does `CONFIG_DIR` — which also means the suite no longer picks up whichever plugins the developer happens to have installed, so a local run and CI exercise the same code. `87dc0e4` is a separate product fix this surfaced: `LocalDatabase.is_odoo` checks that a database exists and then connects to it, and any process can drop it in between — `odev list` inspects every database in turn and would fail outright because one went away. Interrupt handling is covered by `tests/tests/common/test_interrupts.py`: odev captures `SIGINT` around every query to cancel just that statement, so a `Ctrl+C` was previously swallowed and the run carried on. Letting it through instead abandons the connection mid-statement, so the interrupt is recorded and acted upon at the next test boundary. ## 4. Connection lifetime — `47499cb`, `cd07007` Both database context managers built a **second, unconnected** connector to close instead of the one they had connected, so `disconnect()` did nothing and the connection stayed open until the garbage collector got to it: ```python def __enter__(self): self.connector = self._connector_class(self.name).__enter__() # connector A, connected return self def __exit__(self, *args): self._connector_class(self.name).__exit__(*args) # connector B, never connected ``` `ensure_connected` runs every database method inside its own block and those blocks nest — `is_odoo` opens one and then calls `table_exists`, which opens another — so this meant a fresh backend per call. Closing the right connector is **not enough on its own**: an inner block would close the connection the enclosing one is still using. The blocks are now reentrant and share a single connector, counted in `PostgresConnectorMixin` so both classes get the same behaviour. The datastore holds its connection instead of reopening it per read — every command reads it and it lives as long as the process, which is not true of the databases odev walks through for `list` or `delete`. A connection pool keyed per database was considered and set aside: `list --all` and `delete --expression` touch **every** database on the server briefly, so a per-database pool would hold one idle backend per Odoo database until the process ends — the very exhaustion this fixes — unless it also grew a global cap and idle eviction. Measured over a full suite run: | | before | after | |---|---|---| | peak backends held | 42 | **3** | | mean backends held | 8.7 | **0.8** | | suite duration | 54.8s | **33.6s** | | three concurrent suites | died on `max_connections` | **249 passed each**, peak 7 backends | The speedup was not the goal — it is what a backend fork plus an authentication round-trip per query costs. ## Coverage | Module | Before | After | |---|---|---| | `common/string.py` | 85% | **100%** | | `common/version.py` | 96% | **100%** | | `common/postgres.py` | 81% | **93%** | | `common/connectors/git.py` | 34% | **40%** | | **Total** | **64%** | **65%** | ## Verification - `pytest tests` — **249 passed**, from 242 on the first revision - Two and three concurrent suites — **249 passed each**, repeatedly, leaving zero directories and zero databases behind - `SIGINT`, `SIGTERM` and `SIGKILL` mid-run — each verified to leave nothing behind, the last one via the next run's sweep - `odev list --all`, `odev history`, `odev version` — smoke-checked, no connections surviving the process - `pre-commit run --all-files` — clean - `basedpyright` — 3 errors, all pre-existing on `beta`; **0 new** ## Notes for reviewers - `odev/_version.py` is bumped once, to `4.29.10`. `origin/beta` is at `4.29.9`; PRs #175, #176 and #177 each bump from the same base, so whichever merges second needs a one-line rebase. - `LocalDatabase.connector: PostgresConnector | None = None` was removed as dead — `ConnectorMixin.__init__` overwrites it with the connector *class* at construction, which also meant the `if self.connector is not None` guard in `_restore` never protected anything. It is now the `isinstance` check `drop()` already used. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_012tWyDuUYenE92nsBtG19Du
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Every odev invocation paid ~1.6s of framework overhead before the command it was asked to run even began, and kept the user waiting a further ~0.39s after that command had already printed its result. For a tool run dozens of times a day, that overhead is how fast odev feels — and almost none of it was the command: on
odev version, 0.02s of the 1.59s was the command itself.This PR removes that overhead. Nothing about what odev does changes, only what it loads and when.
Results
odev version, best of 10 alternating runs of this branch andbetaon the same machine (interleaved so that background load affects both equally):betaimport odev.commoninit_framework()+start()Absolute numbers are machine-dependent; the ratio held at 54–59% across every run.
tools/benchmark_startup.pyreproduces this table.One caveat worth stating plainly: the first run after an update or a plugin change rebuilds the command index and costs roughly what
betacosts today (measured 1.014s cold vs 0.558s warm on the same machine). Every run after it is the fast path, and the index rebuilds itself transparently.What changed
Five causes, each paid unconditionally on every run:
1. Telemetry blocked the main thread.
run_command'sfinallyjoined a thread doing an HTTPS round-trip to the telemetry endpoint, i.e. the user watched a finished command for a full RTT. Records are now spooled to~/.config/odev/odev-telemetry.jsonlwhen a command ends and submitted in the background by a later run, which has its own command's runtime to do so. No invocation ever waits on the endpoint.2. The debugger scan spawned two
grepprocesses at import time.debug.pyshelled out over the odev tree and every plugin repo on every run (~0.15s) to decide whether to disable spinners. It now runs on demand and caches its result, keyed on a walk of the source trees that costs ~9ms.3. The framework imported dependencies only a few commands need. PyGithub, paramiko, InquirerPy/prompt_toolkit and black were pulled in by
odev.commonbecause the connector and mixin packages re-exported everything they contained. Those imports moved to their point of use, and the packages resolve names lazily through PEP 562__getattr__(newodev/common/lazy.py). Plugins doingfrom odev.common.connectors import GitConnectorkeep working unchanged, andTYPE_CHECKINGdeclarations keep type checkers and IDE navigation intact.networkxwas only used to topologically sort the plugin dependency graph — replaced by a local Kahn sort. Cycle reporting is byte-identical, including thea → b → aand "depends on itself" phrasings.4. Command discovery executed all 46 command modules just to read their names. So every run paid for every command, plugins included — that is where
copier(~0.14s) andnetworkxcame from. Names, aliases and help texts now live in an index cached in~/.config/odev/odev-commands.json; a command module is imported only when that command actually runs. Plugin patching is replayed from the recorded registration order.The index is invalidated by the odev version, each plugin's version, and a hash over the command files' paths and mtimes — so adding, removing, renaming or editing a command rebuilds it transparently.
Two bugs found on the way
Telemetry.update()never ran. Itslen(_command_stack) != 1guard was evaluated afterrun_commandhad popped the stack, so it always returned early: exit codes and execution times have never reached the endpoint. Now reported (and the exit code is the real one instead of a hardcoded0).console.pycompared a class to the string"checkbox", always false, sobypass_promptnever honoureddefaultsfor checkbox prompts. Fixed as a side effect of passing prompt names as strings.Notes for reviewers
tests/fixtures/capture.pyneeded a fix.CaptureOutputattached its handlers to the loggers that existed when it was entered — which no longer covers command modules imported while a command runs. Four database tests failed until it was switched to capture on the root logger. Production was never affected:logging.basicConfiginstalls root handlers and child loggers propagate to them..ruff.tomlgained atools/*per-file-ignore forT201, since a standalone developer script reporting on stdout is the point. The alternative was nine# noqacomments.odev helpoutput is unchanged. Verified by capturingodev help,odev help -1andodev help <cmd>for 11 commands (including plugin-patchedrun,test,pre-commit) before the change and diffing after — byte-identical.Verification
pytest tests— 150 passed (3 new).pre-commit run --all-files— clean.basedpyright— back to exactly the 3 pre-existing errors, zero new.version,version -p,help,help -1,help <cmd>,list,config, a git-backed command, and every plugin command including the patched ones.tests/tests/common/test_startup_performance.pyguards the structural causes rather than timings, which are too noisy to assert on: the framework must import none of the heavy modules, starting it must import zero command modules, and resolving one command must import only that one. They warm the index up first, since building it is the one run that legitimately imports everything.Follow-up (not in this PR)
odev-plugin-ai-upgrade's__init__.pydoesfrom . import commands, which still dragsnetworkxinto startup (~80ms). Deleting that line in that repo removes the last heavy import; odev discovers commands on its own and does not need it.🤖 Generated with Claude Code
https://claude.ai/code/session_012tWyDuUYenE92nsBtG19Du